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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
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- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 
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Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
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Status 

1)[3 Responsive to communication(s) filed on 22 September 2006 . 
2a)Q This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
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Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
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* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1 ) I3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date 6) Q Other: 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 08-06) 



Office Action Summary 



Part of Paper No./Mail Date 20061030 



Application/Control Number: 10/025,912 
Art Unit: 2626 



Page 2 



DETAILED ACTION 

Response to Amendment 

1 . In response to the office action from 7/7/2006, the applicant has submitted an 
amendment, filed 9/22/2006, amending the claims 1, 5, and 7, while arguing to traverse the art 
rejection based on the amended limitations (Amendment, Page 5). The applicant's arguments 
have been fully considered but are moot with respect to the new grounds of rejection in view of 
Hermes et al ("Image Retrieval for Information Systems, " 1995) and Liu et al (U.S. Patent: 
6,970,860). 

Claim Objections 

2. Claim 3 is objected to because of the following informalities: 

In claim 3, line 3, "the word not representing the color" should be changed to -a word not 
representing the color- in order to provide proper antecedent basis. 
Appropriate correction is required. 

Claim Rejections - 35 USC §103 



3. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-3 and 5-8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hermes et al ("Image Retrieval for Information Systems, " 1995) in view of Liu et al (U.S. 
Patent: 6,970,860). 

With respect to Claims 1, 5, and 7, Hermes discloses: 

A dictionary storing means for storing a dictionary used for processing a natural language 
(stored natural language text descriptions associated with image features, Page 396, Section 2); 

A color/shape threshold storing means for storing color histograms mapped to color 
related words and edge information corresponding to shape related words (color histograms and 
related contour data associated with search terms stored in the IRIS system, Pages 396-399, 
Sections 2. 7-2. 1. 3; and Fig. 2) ; 

A query input means for receiving a query that describes the color and the shape of the 
image by using a natural language (input query in a natural language to the IRIS system, Page 
395, Section 2, Page 404, Section 4; and Fig 8)\ 

An analyzing means for analyzing the query sentence based on the dictionary information 
and generating analyzed words (query processing (analysis) to retrieve an image, Fig. 8; and 
Page 403, Section 3)\ 

A color/shape recognizing means for recognizing whether the analyzed words represent 
the color or the shape (color and shape based query processing, Fig. 8)\ 

A color/shape threshold database constructing means for mapping and storing color 
histograms to color related words and storing edge information corresponding to shape related 
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words (IRIS database that maps image features including shape and color to user concepts in a 
natural language for image retrieval, Pages 395-400, Sections 2-2. 3); 

A color/shape threshold retrieving means for retrieving the color histograms and the edge 
information corresponding to the analyzed words from the color/shape threshold storing means 
and retrieving an image satisfying the retrieved color histograms and edge information (query 
processing that retrieves an image corresponding to color and edge information in the IRIS 
system, Pages 403-404, Section 3; and Fig. 8)\ and 

A retrieving result output means for providing image data searched in the color/shape 
threshold retrieving means (query results and corresponding thumbnail images, Fig. 8). 

Although Hermes discloses the ability of a user to input a natural language query to 
locate a desired image, Hermes does not explicitly note that such a query may be in the form of a 
natural language sentence. Hermes also fails to disclose a means for analyzing such a sentence. 
Liu, however, discloses a process for enabling a user to provide a natural language input to a 
search engine for image retrieval, wherein the input is in the form of a natural language sentence 
(natural language text entry area, Col 5, Lines 45-55). Liu also recites a natural language parser 
that identifies keywords through syntactic and semantic information in the input sentence and 
utilizes the keywords in a matching process to retrieve an image (NL parser, Col. 5, Lines 56-65; 
and the matching of extracted keywords to histogram, shape, and texture information associated 
with an image, Col. 6, Lines 38-58). 

Additionally, with respect to claim 7, Liu recites an image retrieval method 
implementation as a program stored on a computer readable medium (Col. 4, Lines 19-31). 
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Hermes and Liu are analogous art because they are from a similar field of endeavor in 
image retrieval systems. Thus, it would have been obvious to a person of ordinary skill in the art, 
at the time of invention, to modify the teachings of Hermes with the natural language sentence 
parsing elements taught by Liu in order to provide a means for better understanding and 
identifying search terms in an image retrieval process (Liu, Col 5, Lines 56-65). 

With respect to Claims 2, 6, and 8, Liu discloses image annotation when no pertinent 
images are retrieved (Col 9, Lines 18-23; and Col 5, Lines 19-25), while Hermes recites query 
terms related to color and shape as applied to Claim 1. 

With respect to Claim 3, Hermes further discloses: 

The color/shape threshold constructing means maps the word representing the color to the 
color histogram and stores the word representing the color mapped to the color histogram 
(mapped image features associated with a color histogram, Pages 396-397, Section 2.1), the 
word not representing the color but reminding the color along with the corresponding color 
histogram (additional words mapped to a color histogram, Page 403, Section 3) and the edge 
information corresponding to the shape related word (mapped contour (shape) features for image 
retrieval, Pages 398-399, Section 2.L3). 

5. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hermes et al in 
view of Liu et al and further in view of Jain et al (U.S. Patent: 5,983,237). 

With respect to Claim 4, Hermes in view of Liu teaches the image query system utilizing 
color and shape information, as applied to claim 3. Hermes in view of Liu do not specifically 
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disclose the use of a qualification relation, however Jain teaches such a relation (Col 9, Lines 
12-24). 

Hermes, Liu, and Jain are analogous art because they are from a similar field of endeavor 
in image retrieval systems. Thus, it would have been obvious to a person of ordinary skill in the 
art, at the time of invention, to modify the teachings of Hermes in view of Liu with the 
qualification relations taught by Jain in order to eliminate unnecessary visual senses when 
constructing a query (Jain, Col 9, Lines 18-20). 

Conclusion 

6. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Vaithilingam et al (U.S. Patent: 6,411, 724)- teaches descriptors for still images in an 
image retrieval process that include color, shape, and texture. 

Mathias et al ("Comparing the Influence of Color Spaces and Metrics in Content-based 
Image Retrieval, " 1998)- teaches an image retrieval system utilizing color-based queries. 

Zachary et al ("Content Based Image Retrieval Systems, " 1999)- provides an overview of 
several color and shape based image retrieval systems. 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (571) 272-7632. 
The examiner can normally be reached on M-Th, 7:30-5:00, F, 7:30-4, Off Alternate Fridays. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Hudspeth can be reached at (571) 272-7843. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



James S. Wozniak 
10/30/2006 





